Skip to content

Support selecting resources for deletion with bundle deploy --select - #7023

Open
andersrexdb wants to merge 2 commits into
mainfrom
select-deletions
Open

andersrexdb wants to merge 2 commits into
mainfrom
select-deletions

Conversation

@andersrexdb

@andersrexdb andersrexdb commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Resolve --select selectors against the computed plan (Plan.FilterToSelected) instead of the config, so resources removed from config and planned for deletion can be selected like any other. The old ResolveSelect mutator is reduced to recording the select_used telemetry flag (renamed MarkSelectUsed).

When a selected resource is being deleted, the plan's closure follows reverse dependency edges: its dependents are deleted too so none are left dangling, while its own dependencies are left untouched — mirroring bundle destroy. A deployed selection still follows forward edges to pull in its dependencies. Grants and permissions sub-nodes move with their parent in both directions.

Deleting a resource whose dependent is still live (still in config, not itself being deleted) is an error before any API delete requests are sent. An invalid --select (a name in neither config nor state) is rejected before the predeploy script runs or any files are uploaded.

Note: When --plan is used alongside a --select that doesn't match the selectors the plan was built with, a warning is emitted. The plan serializes was_filtered and selectors so a pre-filtered plan file can be passed to bundle deploy --plan without re-specifying --select.

Tests

Added unit tests (TestFilterToSelected_*) and acceptance tests under acceptance/bundle/select.

This pull request and its description were written by Isaac.

@andersrexdb
andersrexdb requested review from a team as code owners October 12, 2026 12:40
@andersrexdb
andersrexdb requested a review from denik October 12, 2026 12:40
@github-actions github-actions Bot added the DABs DABs related issues label Oct 12, 2026
andersrexdb added a commit that referenced this pull request Oct 12, 2026
Add the trailing PR link required by the changelog validation so the
Changelog Preview workflow passes.

Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 3f151b0

Run: 39564913615

Env ❌​FAIL 🔄​flaky ✅​pass 🙈​skip Time
🔄​ aws linux-2core-8gb 3 287 18 5:28
✅​ aws-windows-latest-4core-16gb 292 16 4:29
🔄​ azure linux-2core-8gb 3 286 18 4:59
✅​ azure-windows-latest-4core-16gb 291 16 6:31
❌​ gcp linux-2core-8gb 3 287 18 5:25
🔄​ gcp-windows-latest-4core-16gb 3 289 16 6:03
Test Name aws linux-2core-8gb azure linux-2core-8gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
❌​ TestAccept 🔄​f 🔄​f ❌​F 🔄​f
❌​ TestAccept/bundle/select/deletion_dependents 🔄​f 🔄​f ❌​F 🔄​f
❌​ TestAccept/bundle/select/deletion_dependents/DMS= ✅​p 🔄​f ❌​F 🔄​f
🔄​ TestAccept/bundle/select/deletion_dependents/DMS=true 🔄​f ✅​p ✅​p ✅​p
Top 2 slowest tests (at least 2 minutes):
duration env testname
5:36 azure-windows-latest-4core-16gb TestAccept
3:33 aws-windows-latest-4core-16gb TestAccept

Resolve --select selectors against the computed plan (Plan.FilterToSelected)
instead of the config, so resources removed from config and planned for deletion
can be selected like any other. The old ResolveSelect mutator is reduced to
recording the select_used telemetry flag (renamed MarkSelectUsed).

When a selected resource is being deleted, the plan's closure follows reverse
dependency edges: its dependents are deleted too so none are left dangling, while
its own dependencies are left untouched — mirroring bundle destroy. A deployed
selection still follows forward edges to pull in its dependencies. Grants and
permissions sub-nodes move with their parent in both directions.

Deleting a resource whose dependent is still live (still in config, not itself
being deleted) is an error before any API delete requests are sent. An invalid
--select (a name in neither config nor state) is rejected before the predeploy
script runs or any files are uploaded.

When --plan is used alongside a --select that doesn't match the selectors the
plan was built with, a warning is emitted. The plan serializes was_filtered and
selectors so a pre-filtered plan file can be passed to bundle deploy --plan
without re-specifying --select.

Co-authored-by: Isaac <no-reply@databricks.com>
Comment thread .nextchanges/bundles/select-deletions.md Outdated

# A resource removed from config is selectable by name, like any created or updated
# one. The plan is filtered to just its deletion.
trace $CLI bundle plan --select jobs.gone

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we parametrize selector? $SELECTOR

in test.toml: EnvMatrix.SELECTOR = ["jobs.gone", "gone"]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trace outputs the command so the output would differ.

I added a second plan & deploy for "gone" below.

Resources: 3 created, 0 changed, 0 deleted, 0 unchanged

>>> [CLI] bundle plan --select jobs.producer
Error: cannot delete jobs.producer: jobs.consumer still references it; remove the reference or also remove jobs.consumer from the config

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not this configuration fail even without --select? There is a reference to a missing resource, the whole config is not valid.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's invalid, but validate passes and a plain deploy deletes producer and then fails midway without --select.

This PR makes --select fail cleanly before deleting anything, should we do the same without --select as separate PR?

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions

Copy link
Copy Markdown
Contributor

Commit: 3f151b0b8 | Workflow run

Regression test report

Selected 6 changed tests (limit=20, 0 not selected)

12 tests: regression=6 unreleased=0 coverage=6 failing=0

test branch main (0a32c93) latest (v1.21.0)
TestAccept/bundle/select/deletion/DMS= ✅ ❌ ❌
TestAccept/bundle/select/deletion/DMS=true ✅ ❌ ❌
TestAccept/bundle/select/deletion_dependents/DMS= ✅ ❌ ❌
TestAccept/bundle/select/deletion_dependents/DMS=true ✅ ❌ ❌
TestAccept/bundle/select/deletion_saved_plan/DMS= ✅ ❌ ❌
TestAccept/bundle/select/deletion_saved_plan/DMS=true ✅ ❌ ❌
TestAccept/bundle/select/invalid_no_upload/DMS= ✅ ✅ ✅
TestAccept/bundle/select/invalid_no_upload/DMS=true ✅ ✅ ✅
TestAccept/bundle/select/missing/DMS= ✅ ✅ ✅
TestAccept/bundle/select/missing/DMS=true ✅ ✅ ✅
TestAccept/bundle/select/redeploy_unqualified/DMS= ✅ ✅ ✅
TestAccept/bundle/select/redeploy_unqualified/DMS=true ✅ ✅ ✅
TestAccept/bundle/select/deletion/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/select/deletion/output.txt
        +++ /tmp/TestAcceptbundleselectdeletionDMS=2513370348/002/output.txt
        @@ -9,68 +9,17 @@
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
         
         >>> [CLI] bundle plan --select jobs.gone
        -delete jobs.gone
        +Error: no such resource: jobs.gone
         
        -Plan: 0 to add, 0 to change, 1 to delete, 0 unchanged, 1 not selected
         
        -=== bundle deploy --select jobs.gone
        -
        ->>> [CLI] bundle deploy --select jobs.gone
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default/files...
        -Deleted jobs.gone
        -Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 1 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [GONE_ID]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
        -=== Redeploy gone, then select it by unqualified name
        -
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default/files...
        -Created jobs.gone
        -Files: 1 uploaded, 0 deleted
        -Resources: 1 created, 0 changed, 0 deleted, 1 unchanged
        -
        ->>> [CLI] bundle plan --select gone
        -delete jobs.gone
        -
        -Plan: 0 to add, 0 to change, 1 to delete, 0 unchanged, 1 not selected
        -
        -=== bundle deploy --select gone
        -
        ->>> [CLI] bundle deploy --select gone
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default/files...
        -Deleted jobs.gone
        -Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 1 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [GONE_ID_2]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
         === Destroy
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
        +  delete resources.jobs.gone
           delete resources.jobs.keep
         
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 2 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/select/deletion/DMS=true: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/select/deletion/output.txt
        +++ /tmp/TestAcceptbundleselectdeletionDMS=true1725953165/002/output.txt
        @@ -9,68 +9,17 @@
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
         
         >>> [CLI] bundle plan --select jobs.gone
        -delete jobs.gone
        +Error: no such resource: jobs.gone
         
        -Plan: 0 to add, 0 to change, 1 to delete, 0 unchanged, 1 not selected
         
        -=== bundle deploy --select jobs.gone
        -
        ->>> [CLI] bundle deploy --select jobs.gone
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default/files...
        -Deleted jobs.gone
        -Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 1 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [GONE_ID]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
        -=== Redeploy gone, then select it by unqualified name
        -
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default/files...
        -Created jobs.gone
        -Files: 1 uploaded, 0 deleted
        -Resources: 1 created, 0 changed, 0 deleted, 1 unchanged
        -
        ->>> [CLI] bundle plan --select gone
        -delete jobs.gone
        -
        -Plan: 0 to add, 0 to change, 1 to delete, 0 unchanged, 1 not selected
        -
        -=== bundle deploy --select gone
        -
        ->>> [CLI] bundle deploy --select gone
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default/files...
        -Deleted jobs.gone
        -Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 1 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [GONE_ID_2]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
         === Destroy
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
        +  delete resources.jobs.gone
           delete resources.jobs.keep
         
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/select-del-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 2 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/select/deletion_dependents/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/select/deletion_dependents/output.txt
        +++ /tmp/TestAcceptbundleselectdeletion_dependentsDMS=1747934060/002/output.txt
        @@ -10,14 +10,12 @@
         Resources: 3 created, 0 changed, 0 deleted, 0 unchanged
         
         >>> [CLI] bundle plan --select jobs.producer
        -Error: cannot delete jobs.producer: jobs.consumer still references it; remove the reference or also remove jobs.consumer from the config
        +Error: no such resource: jobs.producer
         
         
         >>> [CLI] bundle deploy --select jobs.producer
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-deldep-[UNIQUE_NAME]/default/files...
        -Error: cannot delete jobs.producer: jobs.consumer still references it; remove the reference or also remove jobs.consumer from the config
        +Error: no such resource: jobs.producer
         
        -Files: 1 uploaded, 0 deleted
         
         >>> print_requests.py --sort //jobs/delete --keep
         
        @@ -28,44 +22,18 @@
         Plan: 0 to add, 1 to change, 1 to delete, 1 unchanged
         
         >>> [CLI] bundle plan --select jobs.producer
        -delete jobs.consumer
        -delete jobs.producer
        +Error: no such resource: jobs.producer
         
        -Plan: 0 to add, 0 to change, 2 to delete, 0 unchanged, 1 not selected
         
        -=== bundle deploy --select jobs.producer
        -
        ->>> [CLI] bundle deploy --select jobs.producer
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-deldep-[UNIQUE_NAME]/default/files...
        -Deleted jobs.consumer
        -Deleted jobs.producer
        -Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 2 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [PRODUCER_ID]
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [CONSUMER_ID]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
         === Destroy
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
        +  delete resources.jobs.consumer
           delete resources.jobs.keep
        +  delete resources.jobs.producer
         
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/select-deldep-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 3 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/select/deletion_dependents/DMS=true: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/select/deletion_dependents/output.txt
        +++ /tmp/TestAcceptbundleselectdeletion_dependentsDMS=true1388900728/002/output.txt
        @@ -10,14 +10,12 @@
         Resources: 3 created, 0 changed, 0 deleted, 0 unchanged
         
         >>> [CLI] bundle plan --select jobs.producer
        -Error: cannot delete jobs.producer: jobs.consumer still references it; remove the reference or also remove jobs.consumer from the config
        +Error: no such resource: jobs.producer
         
         
         >>> [CLI] bundle deploy --select jobs.producer
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-deldep-[UNIQUE_NAME]/default/files...
        -Error: cannot delete jobs.producer: jobs.consumer still references it; remove the reference or also remove jobs.consumer from the config
        +Error: no such resource: jobs.producer
         
        -Files: 1 uploaded, 0 deleted
         
         >>> print_requests.py --sort //jobs/delete --keep
         
        @@ -28,44 +22,18 @@
         Plan: 0 to add, 1 to change, 1 to delete, 1 unchanged
         
         >>> [CLI] bundle plan --select jobs.producer
        -delete jobs.consumer
        -delete jobs.producer
        +Error: no such resource: jobs.producer
         
        -Plan: 0 to add, 0 to change, 2 to delete, 0 unchanged, 1 not selected
         
        -=== bundle deploy --select jobs.producer
        -
        ->>> [CLI] bundle deploy --select jobs.producer
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-deldep-[UNIQUE_NAME]/default/files...
        -Deleted jobs.consumer
        -Deleted jobs.producer
        -Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 2 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [PRODUCER_ID]
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [CONSUMER_ID]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
         === Destroy
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
        +  delete resources.jobs.consumer
           delete resources.jobs.keep
        +  delete resources.jobs.producer
         
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/select-deldep-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 3 deleted
        +
        +Exit code: 1
        
FAIL

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/select/deletion_saved_plan/DMS=: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/select/deletion_saved_plan/output.txt
        +++ /tmp/TestAcceptbundleselectdeletion_saved_planDMS=2698323158/002/output.txt
        @@ -7,32 +7,17 @@
         Created jobs.keep
         Files: 3 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        +Error: no such resource: jobs.gone
         
        -=== bundle deploy --plan (no --select)
         
        ->>> [CLI] bundle deploy --plan plan.json
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-plan-[UNIQUE_NAME]/default/files...
        -Deleted jobs.gone
        -Files: 2 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 1 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [GONE_ID]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
         === Destroy
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
        +  delete resources.jobs.gone
           delete resources.jobs.keep
         
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/select-del-plan-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 2 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

TestAccept/bundle/select/deletion_saved_plan/DMS=true: branch ✅ | main (0a32c93) ❌ | latest (v1.21.0) ❌

main (0a32c93):

acceptance_test.go:1307: Diff:
        --- bundle/select/deletion_saved_plan/output.txt
        +++ /tmp/TestAcceptbundleselectdeletion_saved_planDMS=true2313870544/002/output.txt
        @@ -7,32 +7,17 @@
         Created jobs.keep
         Files: 3 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        +Error: no such resource: jobs.gone
         
        -=== bundle deploy --plan (no --select)
         
        ->>> [CLI] bundle deploy --plan plan.json
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/select-del-plan-[UNIQUE_NAME]/default/files...
        -Deleted jobs.gone
        -Files: 2 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 1 deleted, 0 unchanged, 1 not selected
        -
        ->>> print_requests.py --sort //jobs/delete
        -{
        -  "method": "POST",
        -  "path": "/api/2.2/jobs/delete",
        -  "body": {
        -    "job_id": [GONE_ID]
        -  }
        -}
        -
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        -
         === Destroy
         >>> [CLI] bundle destroy --auto-approve
         The following resources will be deleted:
        +  delete resources.jobs.gone
           delete resources.jobs.keep
         
         All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/select-del-plan-[UNIQUE_NAME]/default
         
        -Destroy: 1 deleted
        +Destroy: 2 deleted
        +
        +Exit code: 1

latest (v1.21.0): same output as main (0a32c93)

@andersrexdb
andersrexdb requested a review from denik October 21, 2026 08:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants